Agent-based simulation and child protection systems: Rationale, implementation, and verification.

BACKGROUND AND OBJECTIVE: Simulation models are an important tool used in health care and other disciplines to support operational research and decision-making. In the child protection literature, simulation models are an under-utilized source of research evidence. PARTICIPANTS AND SETTING: In this paper, we describe the rationale for and the development of an agent-based simulation of a child protection system in the US. Using the investigation, prevention service, and placement histories of 600,000 children served in an urban child welfare system, we walk the reader through the development of a prototype known as OSPEDALE. METHODS: The governing equations built into OSPEDALE probabilistically simulate the onset of investigations. Then, drawing from empirical survival distributions, the governing equations trace the probability of subsequent interactions with the system (recurrence of maltreatment, service referrals, and placement) conditional on the characteristics of children, their assessed risk level, and prior child protection system involvement. RESULTS: As an initial test of OSPEDALE's utility, we compare empirical admission counts with counts generated from OSPEDALE. Though the verification step is admittedly simple, the comparison shows that OSPEDALE replicates the empirical count of new admissions closely enough to justify further investment in OSPEDALE. CONCLUSIONS: Management of public child protection systems is increasingly research evidence-dependent. The emphasis on research evidence as a decision-support tool has elevated evidence acquired through randomized clinical trials. Though important, the evidence from clinical trials represents only one type of research evidence. Properly specified, simulation models are another source of evidence with real-world relevance.

Impact of changes in protective behaviors and out-of-household activities by age on COVID-19 transmission and hospitalization in Chicago, Illinois.

PURPOSE: Even with an efficacious vaccine, protective behaviors (social distancing, masking) are essential for preventing COVID-19 transmission and could become even more important if current or future variants evade immunity from vaccines or prior infection. METHODS: We created an agent-based model representing the Chicago population and conducted experiments to determine the effects of varying adult out-of-household activities (OOHA), school reopening, and protective behaviors across age groups on COVID-19 transmission and hospitalizations. RESULTS: From September-November 2020, decreasing adult protective behaviors and increasing adult OOHA both substantially impacted COVID-19 outcomes; school reopening had relatively little impact when adult protective behaviors and OOHA were maintained. As of November 1, 2020, a 50% reduction in young adult (age 18-40) protective behaviors resulted in increased latent infection prevalence per 100,000 from 15.93 (IQR 6.18, 36.23) to 40.06 (IQR 14.65, 85.21) and 19.87 (IQR 6.83, 46.83) to 47.74 (IQR 18.89, 118.77) with 15% and 45% school reopening. Increasing adult (age ≥18) OOHA from 65% to 80% of prepandemic levels resulted in increased latent infection prevalence per 100,000 from 35.18 (IQR 13.59, 75.00) to 69.84 (IQR 33.27, 145.89) and 38.17 (IQR 15.84, 91.16) to 80.02 (IQR 30.91, 186.63) with 15% and 45% school reopening. Similar patterns were observed for hospitalizations. CONCLUSIONS: In areas without widespread vaccination coverage, interventions to maintain adherence to protective behaviors, particularly among younger adults and in out-of-household settings, remain a priority for preventing COVID-19 transmission.

Building and experimenting with an agent-based model to study the population-level impact of CommunityRx, a clinic-based community resource referral intervention.

CommunityRx (CRx), an information technology intervention, provides patients with a personalized list of healthful community resources (HealtheRx). In repeated clinical studies, nearly half of those who received clinical "doses" of the HealtheRx shared their information with others ("social doses"). Clinical trial design cannot fully capture the impact of information diffusion, which can act as a force multiplier for the intervention. Furthermore, experimentation is needed to understand how intervention delivery can optimize social spread under varying circumstances. To study information diffusion from CRx under varying conditions, we built an agent-based model (ABM). This study describes the model building process and illustrates how an ABM provides insight about information diffusion through in silico experimentation. To build the ABM, we constructed a synthetic population ("agents") using publicly-available data sources. Using clinical trial data, we developed empirically-informed processes simulating agent activities, resource knowledge evolution and information sharing. Using RepastHPC and chiSIM software, we replicated the intervention in silico, simulated information diffusion processes, and generated emergent information diffusion networks. The CRx ABM was calibrated using empirical data to replicate the CRx intervention in silico. We used the ABM to quantify information spread via social versus clinical dosing then conducted information diffusion experiments, comparing the social dosing effect of the intervention when delivered by physicians, nurses or clinical clerks. The synthetic population (N = 802,191) exhibited diverse behavioral characteristics, including activity and knowledge evolution patterns. In silico delivery of the intervention was replicated with high fidelity. Large-scale information diffusion networks emerged among agents exchanging resource information. Varying the propensity for information exchange resulted in networks with different topological characteristics. Community resource information spread via social dosing was nearly 4 fold that from clinical dosing alone and did not vary by delivery mode. This study, using CRx as an example, demonstrates the process of building and experimenting with an ABM to study information diffusion from, and the population-level impact of, a clinical information-based intervention. While the focus of the CRx ABM is to recreate the CRx intervention in silico, the general process of model building, and computational experimentation presented is generalizable to other large-scale ABMs of information diffusion.

Impact of a Low-Intensity Resource Referral Intervention on Patients' Knowledge, Beliefs, and Use of Community Resources: Results from the CommunityRx Trial.

BACKGROUND: Connecting patients to community-based resources is now a cornerstone of modern healthcare that supports self-management of health. The mechanisms that link resource information to behavior change, however, remain poorly understood. OBJECTIVE: To evaluate the impact of CommunityRx, an automated, low-intensity resource referral intervention, on patients' knowledge, beliefs, and use of community resources. DESIGN: Real-world controlled clinical trial at an urban academic medical center in 2015-2016; participants were assigned by alternating week to receive the CommunityRx intervention or usual care. Surveys were administered at baseline, 1 week, 1 month, and 3 months. PARTICIPANTS: Publicly insured adults, ages 45-74 years. INTERVENTION: CommunityRx generated an automated, personalized list of resources, known as HealtheRx, near each participant's home using condition-specific, evidence-based algorithms. Algorithms used patient demographic and health characteristics documented in the electronic health record to identify relevant resources from a comprehensive, regularly updated database of health-related resources in the study area. MAIN MEASURES: Using intent-to-treat analysis, we examined the impact of HealtheRx referrals on (1) knowledge of the most commonly referred resource types, including healthy eating classes, individual counseling, mortgage assistance, smoking cessation, stress management, and weight loss classes or groups, and (2) beliefs about having resources in the community to manage health. KEY RESULTS: In a real-world controlled trial of 374 adults, intervention recipients improved knowledge (AOR = 2.15; 95% CI, 1.29-3.58) and beliefs (AOR = 1.68; 95% CI, 1.07-2.64) about common resources in the community to manage health, specifically gaining knowledge about smoking cessation (AOR = 2.76; 95% CI, 1.07-7.12) and weight loss resources (AOR = 2.26; 95% CI 1.05-4.84). Positive changes in both knowledge and beliefs about community resources were associated with higher resource use (P = 0.02). CONCLUSIONS: In a middle-age and older population with high morbidity, a low-intensity health IT intervention to deliver resource referrals promoted behavior change by increasing knowledge and positive beliefs about community resources for self-management of health. NIH TRIAL REGISTRY: NCT02435511.